The Accuracy and Efficiency of a Reference-Based Adaptive Selection Algorithm for Comparative Judgment

Several studies have proven that comparative judgment (CJ) is a reliable and valid assessment method for variety of competences, expert assessment, peer CJ emerging as possible approach to help maintain standards over time. For consecutive pairs student works (representations) assessors are asked judge which representation better. It has been shown random construction leads very inefficient assessments, requiring lot pairwise comparisons reach results. Some adaptive selection algorithms using information from previous were proposed increase the efficiency CJ. These appear however artificially inflate reliability results through increasing spread The current article proposes new algorithm previously calibrated reference set. Using set should eliminate inflation. In real sets different reliability, in simulation study, it this more efficient without reducing accuracy standard deviation As consequence, reference-based produces high correct values an manner.